Papers with Contextual embeddings
On the Language Neutrality of Pre-trained Multilingual Representations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on cross-linguality of contextual embeddings . however, they are only moderately language-neutral by default . |
| Approach: | They propose to use unsupervised centering and fitting an explicit projection on parallel data to achieve stronger language neutrality. |
| Outcome: | The proposed model outperforms existing models on XNLI and NER tasks. |
The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Contextual embeddings have shown state-of-the-art performance for various tasks such as question answering, sentiment analysis, and textual similarity. |
| Approach: | They propose to integrate transformer-based neural language models into their models to achieve relative improvements of up to 36% on granular tasks. |
| Outcome: | The proposed model outperforms baselines for more granular tasks while outperforming TF-IDF for more complex tasks. |
Does BERT Recognize an Agent? Modeling Dowty’s Proto-Roles with Contextual Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Contextual embeddings build multidimensional representations of word tokens based on their context of occurrence. |
| Approach: | They propose to map the verb embeddings to an interpretable space of semantic properties built from a linguistic dataset and test their ability to model the semantic properties of the agent of the verbs participating in the alternation. |
| Outcome: | The proposed models can model the semantic properties of the verbs participating in the so-called causative alternation. |
A Synset Relation-enhanced Framework with a Try-again Mechanism for Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing sense embeddings fail to embed sense knowledge in semantic networks. |
| Approach: | They propose a Synset Relation-Enhanced Framework that leverages sense relations for sense embedding enhancement and a try-again mechanism that implements WSD again. |
| Outcome: | The proposed system outperforms knowledge-based systems with 20% SemCor data on all-words and lexical datasets. |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |